Qdrant is a high-performance vector search engine and database that offers a dependable and production-ready service with a simple API for storing, searching, and managing vectors and other metadata. Qdrant is intended to provide extended filtering, making it particularly appropriate for applications requiring neural network or semantic-based matching, faceted search, and other related use cases.
Qdrant is written in the Rust programming language, which promises unrivalled speed, performance, and reliability even under extreme load. Qdrant’s benchmarks indicate its capacity to efficiently and swiftly handle massive amounts of high-dimensional data.
Table of Contents
Qdrant enables you to transform embeddings or neural network encoders into full-fledged apps for matching, searching, recommending, and much more. Its user-friendly API and extensive search features make it simple to create complicated similarity search apps without worrying about the underlying infrastructure.
Local Installation
Install Qdrant client using pip.
pip install qdrant-clientThe Python client allows you to run the same code in local mode without launching the Qdrant server. Simply put, initialise client as follows:
from qdrant_client import QdrantClient
client = QdrantClient(path="path/to/db")Qdrant Install using Docker
The simplest approach to get started with Qdrant is to run it from a Docker image. DockerHub always has the most recent versions.
sudo docker infoPull the image for Docker
docker pull qdrant/qdrantRun the container
docker run -p 6333:6333
-v $(pwd)/path/to/data:/qdrant/storage
qdrant/qdrantThis command launches a Qdrant instance using the default settings. It will save all data in the directory ./path/to/data.
Now Qdrant should be accessible at localhost:6333.
Connect to Qdrant cloud
You can register and use Qdrant Cloud to get a free tier.
Once you have your cluster and API key, you can connect to it like this:
from qdrant_client import QdrantClient
qdrant_client = QdrantClient(
url="https://xxxxxx-xxxxx-xxxxx-xxxx-xxxxxxxxx.us-east.aws.cloud.qdrant.io:6333",
api_key="<your-api-key>",
)Examples
Create a new collection
from qdrant_client.models import Distance, VectorParams
client.recreate_collection(
collection_name="my_collection",
vectors_config=VectorParams(size=100, distance=Distance.COSINE),
)Insert vectors into a collection
import numpy as np
from qdrant_client.models import PointStruct
vectors = np.random.rand(100, 100)
client.upsert(
collection_name="my_collection",
points=[
PointStruct(
id=idx,
vector=vector.tolist(),
payload={"color": "red", "rand_number": idx % 10}
)
for idx, vector in enumerate(vectors)
]
)Search for similar vectors
query_vector = np.random.rand(100)
hits = client.search(
collection_name="my_collection",
query_vector=query_vector,
limit=5 # Return 5 closest points
)Search for similar vectors with filtering condition
from qdrant_client.models import Filter, FieldCondition, Range
hits = client.search(
collection_name="my_collection",
query_vector=query_vector,
query_filter=Filter(
must=[ # These conditions are required for search results
FieldCondition(
key='rand_number', # Condition based on values of `rand_number` field.
range=Range(
gte=3 # Select only those results where `rand_number` >= 3
)
)
]
),
limit=5 # Return 5 closest points
)gRPC
To enable (typically, much faster) collection uploading with gRPC, use the following initialization:
from qdrant_client import QdrantClient
client = QdrantClient(host="localhost", grpc_port=6334, prefer_grpc=True)Async client
Raw autogenerated clients support async methods. Normally, you don’t need to use them directly, but if you need extra performance, you can.
Example of using raw async gRPC client:
from qdrant_client import QdrantClient, grpc
client = QdrantClient(prefer_grpc=True, timeout=3.0)
grpc_collections = client.async_grpc_collections
res = await grpc_collections.List(grpc.ListCollectionsRequest(), timeout=1.0)Development
This project uses git hooks to run code formatters.
Install pre-commit with pip3 install pre-commit and set up hooks with pre-commit install.
REST
Online OpenAPI 3.0 documentation is available here. OpenAPI makes it easy to generate a client for virtually any framework or programming language. You can also download raw OpenAPI definitions.
Also, Read Google Search Engine and OpenAI’s ChatGPT.
Features
Filtering and Payload
JSON payloads can be associated with vectors in Qdrant, which provides both storing and filtering depending on payload values. Unlike Elastic Search post-filtering, it enables various combinations of should, must, and must_not conditions, ensuring recovery of all relevant vectors.
The vector payload supports a wide range of data kinds and query conditions, such as text matching, numerical ranges, geo-locations, and others. These filtering conditions give you the ability to build unique business logic on top of similarity matching.
Query Planning and Payload Indexes
To optimise query execution, the query planner makes use of cached payload information. Smaller search spaces constrained by filters, for example, may benefit from full brute force over an index.
Qdrant gives higher search performance on modern hardware by utilising modern CPU x86-x64 architectures.
Qdrant supports data persistence by confirming updates, even during power interruptions. The update log records all operations, allowing for easy reconstruction of the most recent database state.
As of v0.8.0, Qdrant supports distributed deployment. Multiple Qdrant machines form a cluster for horizontal scaling, coordinated through the Raft protocol.
Qdrant operates independently of external databases or orchestration controllers, which simplifies configuration.
This article is to help you learn about Qdrant. We trust that it has been helpful to you. Please feel free to share your thoughts and feedback in the comment section below.
Leave a Reply
You must be logged in to post a comment.